Systems and methods for optimizing a model file

ABSTRACT

Systems and methods for optimizing a model file include an occlusion reduction process. The occlusion reduction process includes acts of: (i) identifying points on a mesh section of a model, (ii) casting a set of rays toward the points, (iii) determining a number of rays that reach the points without being occluded by a portion of the model or a portion of another object, (iv) determining occlusion values for the points based on the number of rays that reach the points without being occluded; (v) determining whether the occlusion values for the points satisfy a predetermined occlusion threshold value; (vi) removing the mesh section from the model in response to determining that the occlusion values satisfy the predetermined occlusion threshold value, and (vii) refraining from removing the mesh section from the model in response to determining that the occlusion values fail to satisfy the predetermined occlusion threshold value.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/882,190 filed May 22, 2020, the disclosure of which is incorporatedherein by reference in its entirety.

BACKGROUND Technical Field

This disclosure generally relates to extended reality (XR) technologies.More specifically, the present disclosure relates to the optimization ofmodel files for display in XR implementations.

Related Technology

In recent times, advances in computing power have enabled computingsystems to provide new and varied experiences to users. One suchcategory of new and varied user experiences relates to the areas ofcomputer-implemented realities. For example, augmented reality (AR) is alive, direct, or indirect view of a physical, real-world environmentwhose elements are augmented (or supplemented) by computer-generatedsensory input such as sound, video, graphics, or GPS data. Augmentedreality utilizes a user's existing reality and adds to it via acomputing device, display, or projector of some sort. For example, manymobile electronic devices, such as smartphones and tablets, can overlaydigital content into the user's immediate environment through use of thedevice's camera feed and associated viewer. Thus, for example, a usercould view the user's real-world environment through the display of amobile electronic device while virtual objects are also being displayedon the display, thereby giving the user the sensation of having virtualobjects together with the user in a real-world environment. CustomAR-enabled headsets can also be used.

Virtual reality (VR) is another example of a computer-implementedreality. In general, VR refers to computer technologies that use realityheadsets and/or other peripheral devices to generate sounds, images, andother sensations that replicate a real environment or that form animaginary world. Virtual reality immerses a user in an entirely virtualexperience and allows the user to interact with the virtual environment.A true VR environment engages all five senses (taste, sight, smell,touch, sound), but as used herein, the term “virtual reality” or “VR” isintended to include those computer-implemented realities that engage atleast the user's sense of sight and that do not display the user'simmediate visual environment.

Another example of a computer-implemented reality is a hybrid realitycalled mixed reality (MR). Mixed reality represents the merging of realand virtual worlds to produce new environments and visualizations wherephysical and digital objects co-exist and interact in real time. Many MRimplementations place new imagery within a real space and often do so insuch a way that the new imagery can interact—to an extent—with what isreal in the physical world. For example, in the context of MR, a usermay view a white board through an MR-enabled headset and use a digitallyproduced pen (or even a capped physical pen) to write on the whiteboard. In the physical world, no writing appears on the white board, butwithin the MR environment, the user's interaction with a real-worldobject caused a digital representation of the writing to appear on thewhite board. In MR systems, some synthetic content can react and/orinteract with the real-world content in real time.

An umbrella term, namely extended reality (XR), incorporates each of theforms of computer-generated reality content—AR, VR, and MR. As usedherein, the term “extended reality” or “XR” refers generally to all realand virtual combined environments and human-machine interactionsgenerated by computer technology and wearables. Extended realityincludes all its descriptive forms, such as digital representations madeor displayed within AR, VR, or MR.

Extended reality experiences are a fairly recent development, and as aresult, many different entities are each developing their own XRrendering platforms and corresponding model or scene files to be used onthose platforms. For example, Apple Corporation has a rendering platformknown as ARKit, which uses AR object files in the .usdz format and/orthe .reality format to provide AR experiences to a user. AndroidCorporation provides the ARCore platform and uses the .gltf formatand/or the .glb format. Other platforms and/or implementations use anyof the foregoing and/or the .obj, .fbx, .dae, and/or .sfb format toprovide AR user experiences to a user.

Experienced and novice developers often create AR experiences fromexisting 3D model files, which are then saved in an appropriate ARobject file format for presentation on an end user AR platform. 3D modelfiles typically include large amounts of object data (e.g., vertices,edges, faces, textures, NURBS modeling data) in order to provide arealistic representation of the modeled content. Consequently,high-quality 3D model files and AR scene files that implementhigh-quality 3D model files are quite large (e.g., on the order ofhundreds of megabytes in size).

In many instances, presenting XR content to a user is a computationallyintensive process. For example, presenting AR content often involvesdepth detection, surface reconstruction, tracking of the viewing device(e.g., a smart phone, tablet, laptop, or head-mounted display (HMD)),and/or rendering 3D models anchored to real-world objects (which aresometimes moving) relative to the viewing device. AR platforms varywidely in their hardware specifications and capabilities, ranging fromlower-end handheld devices (e.g., smartphones) to dedicated AR HMDs.Because of the computational expense involved with presenting ARcontent, many consumer AR platforms (e.g., handheld devices) are unableto present AR experiences that include detailed 3D models that are largein file size. In some instances, consumer AR platforms fail to provide aseamless AR experience or the AR platform crashes when attempting torender detailed 3D models of an AR scene file. Some AR platforms becomedamaged when attempting to render complex AR scenes.

This restricts the ability of users to share AR experiences with otherusers and prevents experienced and novice developers from being able toreach large swaths of consumers with their AR content. Consequently,developers typically create AR experiences that are geared to high-endAR platforms (e.g., AR HMDs, tablets, or desktop computers with highprocessing capabilities, etc.). Although some generaloptimization/compression methods exist for 3D models (e.g., geometriccompression), these compression methods often result in loss of visualfidelity and fail to account for special circumstances associated withpresenting AR environments (e.g., presenting the model in conjunctionwith a real-world environment).

Accordingly, there are a number of disadvantages with the optimizationof model files for display in XR that can be addressed.

The subject matter claimed herein is not limited to embodiments thatsolve any disadvantages or that operate only in environments such asthose described above. Rather, this background is only provided toillustrate one exemplary technology area where some embodimentsdescribed herein may be practiced.

BRIEF SUMMARY

Implementations of the present disclosure solve one or more of theforegoing or other problems in the art with the optimization of modelfiles for display in XR. In particular, one or more implementations caninclude a method for optimizing a model file, such as a 3D model file,or an AR scene or model file that includes 3D models.

In some embodiments, a method for optimizing a model file includes anocclusion reduction process. The occlusion reduction process may includeacts of: (i) identifying one or more points on a mesh section of amodel, (ii) determining an occlusion value for each of the one or morepoints on the surface of the mesh section by performing a ray castrendering of each of the one or more points, (iii) determining whetherthe one or more occlusion values meet or exceed a predeterminedocclusion threshold value, and (iv) removing the mesh section from themodel.

In some embodiments, the method for optimizing a model file includesother optimization processes, such as an instancing step, a selectivedecimation step, a texture reduction step, and/or a texture formattingstep. An optimized model file may be converted into various 3D modeland/or AR object or scene file formats for use on various AR platforms.

Some embodiments include systems for optimizing a model. In one aspect,a system includes processor(s) and hardware storage device(s) storingcomputer-executable instructions that are operable, when executed by theone or more processors, to configure the system to perform an occlusionreduction process. The inclusion reduction process can includeconfiguring the system to at least (i) identify points on a mesh sectionof a model, (ii) casting a set of rays toward the points, (iii)determine a number of rays that reach the points without being occludedby a portion of the model or a portion of another object, (iv) determineocclusion values for the points based on the number of rays that reachthe points without being occluded; (v) determine whether the occlusionvalues for the points satisfy a predetermined occlusion threshold value;(vi) remove the mesh section from the model in response to determiningthat the occlusion values satisfy the predetermined occlusion thresholdvalue, and (vii) refrain from removing the mesh section from the modelin response to determining that the occlusion values fail to satisfy thepredetermined occlusion threshold value.

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the detaileddescription. This summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an indication of the scope of the claimed subject matter.

Additional features and advantages of the disclosure will be set forthin the description which follows, and in part will be obvious from thedescription, or may be learned by the practice of the disclosure. Thefeatures and advantages of the disclosure may be realized and obtainedby means of the instruments and combinations particularly pointed out inthe appended claims. These and other features of the present disclosurewill become more fully apparent from the following description andappended claims or may be learned by the practice of the disclosure asset forth hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above recited and otheradvantages and features of the disclosure can be obtained, a moreparticular description of the disclosure briefly described above will berendered by reference to specific embodiments thereof, which areillustrated in the appended drawings. It is appreciated that thesedrawings depict only typical embodiments of the disclosure and are nottherefore to be considered to be limiting of its scope.

The disclosure will be described and explained with additionalspecificity and detail through the use of the accompanying drawings inwhich:

FIG. 1 illustrates an example representation of various vertices,surfaces, and edges of a mesh of a model;

FIG. 2 illustrates a conceptual representation of performing ray castrendering on various points of the model;

FIG. 3 illustrates a conceptual representation of performing ray castrendering on various points of the model when the model is in adifferent state;

FIGS. 4A and 4B illustrates conceptual representations of a real-worlddistance associated with a viewing distance and a display device;

FIGS. 5A and 5B illustrate conceptual representations of selectivedecimation and texture reduction processes;

FIG. 6 illustrates an example of a 3D model that includes model objectsthat include identical vertices;

FIG. 7 illustrates an example flow diagram depicting a method foroptimizing a model file;

FIG. 8 illustrates an example flow diagram depicting a method forperforming an occlusion reduction process;

FIG. 9 illustrates an example flow diagram depicting a method forperforming a selective decimation process;

FIG. 10 illustrates an example flow diagram depicting a method forperforming a texture reduction process;

FIG. 11 illustrates an example flow diagram depicting a method forperforming a texture formatting process; and

FIG. 12 illustrates an example flow diagram depicting a method forperforming an instancing process.

DETAILED DESCRIPTION

Before describing various embodiments of the present disclosure indetail, it is to be understood that this disclosure is not limited tothe parameters of the particularly exemplified systems, methods,apparatus, products, processes, and/or kits, which may, of course, vary.Thus, while certain embodiments of the present disclosure will bedescribed in detail, with reference to specific configurations,parameters, components, elements, etc., the descriptions areillustrative and are not to be construed as limiting the scope of theclaimed invention. In addition, any headings used herein are fororganizational purposes only, and the terminology used herein is for thepurpose of describing the embodiments. Neither are not meant to be usedto limit the scope of the description or the claims.

Overview

Experienced and novice developers often create augmented reality (AR)experiences from existing 3D model files, which are then saved in anappropriate AR object file format for presentation on an end user ARplatform. 3D model files typically include large amounts of object data(e.g., vertices, edges, faces, textures, NURBS modeling data) in orderto provide a realistic representation of the modeled content.Consequently, high-quality 3D model files and AR scene files thatimplement high-quality 3D model files are quite large (e.g., on theorder of hundreds of megabytes in size).

In many instances, presenting XR content to a user is a computationallyintensive process. For example, presenting AR content often involvesdepth detection, surface reconstruction, tracking of the viewing device(e.g., a smart phone, tablet, laptop, or head-mounted display (HMD)),and/or rendering 3D models anchored to real-world objects (which aresometimes moving) relative to the viewing device. AR platforms varywidely in their hardware specifications and capabilities, ranging fromlower-end handheld devices (e.g., smartphones) to dedicated AR HMDs.Because of the computational expense involved with presenting ARcontent, many consumer AR platforms (e.g., handheld devices) are unableto present AR experiences that include detailed 3D models that are largein file size. In some instances, consumer AR platforms fail to provide aseamless AR experience or the AR platform crashes when attempting torender detailed 3D models of an AR scene file. Some AR platforms becomedamaged when attempting to render complex AR scenes.

This restricts the ability of users to share AR experiences with otherusers and prevents experienced and novice developers from being able toreach large swaths of consumers with their AR content. Consequently,developers typically create AR experiences that are geared to high-endAR platforms (e.g., AR HMDs, tablets, or desktop computers with highprocessing capabilities, etc.). Although some generaloptimization/compression methods exist for 3D models (e.g., geometriccompression), these compression methods often result in loss of visualfidelity and fail to account for special circumstances associated withpresenting AR environments (e.g., presenting the model in conjunctionwith a real-world environment).

Disclosed embodiments are directed to systems and methods for optimizinga model file includes an occlusion reduction process. The occlusionreduction process may include acts of: i) identifying one or more pointson a mesh section of a model, ii) determining an occlusion value foreach of the one or more points on the surface of the mesh section byperforming a ray cast rendering of each of the one or more points, iii)determining whether the one or more occlusion values meet or exceed apredetermined occlusion threshold value, and iv) removing the meshsection from the model.

Furthermore, in some instances, the method for optimizing a model fileincludes other optimization processes, such as an instancing step, aselective decimation step, a texture reduction step, and/or a textureformatting step. The selective decimation and/or texture reduction stepsmay be based on a real-world distance threshold, that may be based on apredefined viewing distance and a screen pixel size. The method thenincludes outputting an optimized model file. The optimized model filemay be converted into various 3D model and/or AR object or scene fileformats for use on various AR platforms.

Those skilled in the art will appreciate that the disclosed embodimentsmay address many problems associated with the optimization of modelfiles for display in AR environments. For instance, low-end AR platforms(e.g., handheld devices with limited hardware capabilities) may not beable to render AR experiences that include detailed 3D models.Additionally, at least some optimization methods of the presentdisclosure account for the real-world environments in which a user willexperience the 3D models of the AR content, providing optimizationstechniques that are uniquely tailored to AR experiences. Furthermore, atleast some optimization techniques disclosed herein allow for reductionin the file size of 3D model files and/or AR model or scene fileswithout sacrificing visual fidelity. Still furthermore, those skilled inthe art will appreciate that the methods disclosed herein may beimplemented in conjunction with other optimization techniques (e.g.,geometric compression) to further reduce the size of the 3D model filesand/or AR model or scene files (albeit with a loss in visual fidelity).

Disclosed embodiments may allow AR content developers to confidentlydevelop AR content for use on lower-end AR platforms (e.g.,smartphones), thus opening new markets and allowing users to share ARexperiences peer-to-peer without concern that an AR experience will beunrenderable on a peer's AR platform.

As used herein, the terms “optimal,” “optimum,” “optimize,” andderivative forms thereof (e.g., optimization, optimizing) are intendedto connote the transformation and/or change from a base form or state toa new form or state. Such transformation is marked by the new form orstate having at least a marginal improvement/benefit compared to thebase form or state. In other words, the new form or state resulting from“optimization” of the base form or state exists as an at leastmarginally improved base form or state. As a non-limiting example, an“optimized” 3D image file can be generated by reducing the file size ofa base 3D image file. Preferably, and as exemplified by processes andembodiments described herein, an “optimization” step that reduces thefile size can also maintain the perceived visual fidelity of the 3Dimage. The “optimized” 3D file can therefore be said to have beentransformed from a base state to a new state marked by at least amarginal improvement with respect to its storage requirements.

As an additional, non-limiting example, an “optimized” 3D image file canundergo an “optimization” process that reduces the number of renderedvertices while, preferably, maintaining perceptible visual fidelity ofthe rendered 3D image. As such, the “optimized” 3D file can be said tohave been transformed from a base state to a new state marked by atleast a marginal improvement with respect to the efficiency by which itcan be rendered. Such efficiency can be measured, for example, as areduced number of processing cycles required to complete the rendering.Where “optimization” involves the at least a marginal improvement torendering or visualization efficiency, it can also preferably maintainperceptible visual fidelity of the rendered image. As used herein, theterm “perceptible visual fidelity” is intended to encompass thoserenderings whose visible surfaces are either objectively or subjectivelythe same (or substantially similar to one another).

It should be appreciated that these foregoing terms can refer to anynumber of components, implementations, processes, embodiments, states,outputs, and/or other elements that provide or enable such marginalimprovement/benefit, and these terms should not be construed asnecessarily meaning a singular component, implementation, process,embodiment, state, output, and/or other element that provides a highestdegree of improvement/benefit.

Systems and Methods for Optimizing a Model File

FIG. 1 illustrates an example representation of various vertices,surfaces, and edges of a mesh of a model 100. In the exemplaryembodiment shown, model 100 is a 3D model of a piece of furniture, whicha computing system receives as a 3D model file (comprising the 3Dfurniture object) or an AR model or scene file (comprising 3D modelsthat include the 3D furniture object). AR model or scene files take onvarious forms, such as .usdz, .reality, .gltf, .glb, .obj, .fbx, .dae,or .sfb formats. Similarly, 3D model files take on various forms, suchas 3DS, A3D, COB, SCN, C4D, DAE, DXF, IGES, IV, FBX, KMZ, LW, LWB, LWM,LWO, LWS, LXO, MA, MB, MAX, OBJ, PLY, SKP, STEP, STL, WRL, WRL97, VRML,X, X3D, or XS formats. It will be appreciated that the optimizationtechniques disclosed herein are not limited to 3D models of anyparticular 3D model file or AR model or scene file format.

In this regard, at least some of the optimization processes disclosedherein are performed by a computing system that obtains, receives,and/or identifies a 3D model or object that is part of a 3D model fileor an AR model or scene file. Exemplary computing systems are discussedhereinafter.

As shown in FIG. 1, model 100 includes a plurality of mesh sections 120,122, 124, and 126 that form the model 100 (not all mesh sections thatmodel 100 is comprised of are shown, for clarity). The example meshsections 120, 122, 124, and 126 of model 100 shown in FIG. 1 areillustrated as triangular sections, although the disclosed embodimentsare not so limited. For example, a mesh section may take the form of anypolygonal or curved section that forms the mesh of the model 100.

The mesh sections 120, 122, 124, and 126 include vertices, which arepoints shared by three or more adjacently arranged mesh sections ofmodel 100 (see also vertices 532 and 534 of FIGS. 5A and 5B; adjacentlyarranged mesh sections are not shown in FIG. 1). For clarity, only thevertices 130 of mesh section 124 are labeled in FIG. 1. The meshsections 120, 122, 124, and 126 also include edges, which are linesshared by two adjacent mesh sections. For clarity, only the edges 132 ofmesh section 122 are labeled in FIG. 1. The mesh sections 120, 122, 124,and 126 also include surfaces that fill the space between each edge andvertex of a mesh section. For clarity, only the surface 134 of meshsection 120 is labeled in FIG. 1.

Mesh section 120 is shown as being positioned on a top (exposed) side ofa top element 102 of the model 100 (e.g., representing a pillow of anottoman), mesh section 122 is shown as being positioned on a bottom side104 of the model 100 (in dotted lines), mesh section 124 is shown asbeing positioned on an under (covered) side of the top element 102 ofthe model 100 (e.g., on an underside of the pillow of the ottoman, shownin dotted lines), and mesh section 126 is shown as being positioned onthe underside of the top element 102 of the model, but with one edge 136of the mesh section 126 being arranged along a side of the model 100 andtherefore exposed/viewable.

One aspect of optimizing a model file, according to the presentdisclosure, is performing an occlusion reduction process. Wheninteracting with a 3D model on a conventional computer interface, it isoften necessary to preserve model characteristics that are not visiblewhen viewing the model from its exterior (e.g., threads of a screw in anCAD file of an engineering part). However, in many AR scenarios, only afinal visual of an exterior of the model is needed (e.g., when placingfurniture in a real-world environment while shopping). Therefore, in anAR implementation, at least some interior mesh sections of the model maybe removed from the 3D model file, thereby reducing its size andusability in mobile AR experiences.

An occlusion reduction process may include identifying one or morepoints on a surface of a mesh section of the model. As illustrated inFIG. 1, various points of a mesh section of a model 100 may beconsidered when performing an occlusion reduction process. In someinstances, a system identifies a center (or other) point of a surface ofa mesh section, such as center point 144 of surface 134 of mesh section120. In some instances, a system identifies a vertex point of a meshsection, such as any number of the points of vertices 130 of meshsection 124. Furthermore, in some instances, a system identifies one ormore points along an edge of a mesh section, such as center points 142of edges 132 of mesh section 122. Additionally, in some implementations,a system identifies various combinations of surface points, edge points,and/or vertex points as part of the occlusion reduction process, such asa vertex point 146, edge point 148, and surface point 150.

A system may identify any number of points of a mesh section to carryout an occlusion reduction process, however, a greater number of pointsmay be correlated with a more accurate occlusion reduction process(e.g., to avoid over-reduction). For example, in some embodiments, acomputer system identifies a center point of the surface, a center edgepoint for each edge, and all vertex points associated with a meshsection upon which the occlusion reduction process will be performed.

Upon identifying the desired points of one or more mesh sections, thesystem performs ray cast rendering on each of the points. Ray castrendering includes casting various rays (representing, e.g., ambientlight) toward the identified points to determine whether the rays areoccluded by other parts of the model or whether they reach theidentified points. The system determines an occlusion score for each ofthe identified points based on the number of rays that reach theidentified point.

FIG. 2 illustrates a conceptual representation of performing ray castrendering on various points of the model 100. As shown, various rays 160are cast toward the various points identified on mesh sections 120, 122,124, and 126. Although the rays 160 are shown in FIG. 2 as being cast ina particular arrangement, and only a particular number of rays areshown, those skilled in the art will recognize that any configurationand/or number of rays cast toward the various identified points iswithin the scope of this disclosure. By way of non-limiting example, allrays can be cast radially inward toward a center point positioned on oneof the identified points, and the process can be repeated for eachidentified point. Rather than casting rays radially inward, in someinstances, the system casts rays from various point or geometricsources, sequentially or concurrently. In some embodiments, the systemperforms ray cast rendering in only a single iteration, wherein thesystem casts all rays once and assesses which, if any, points werereached by the rays.

In the embodiment illustrated in FIG. 2, the system accounts forenvironmental surfaces when performing ray cast rendering. In someembodiments, the system determines whether the model 100 will be visiblefrom certain viewing angles when implemented in an AR experience. Thismay, for example, be based on a real-world surface to which the model100 will be anchored when implemented in an AR experience (or a surface,whether real or virtual, that the model 100 will be displayed within athreshold proximity to when implemented in an AR environment). Forinstance, FIG. 2 shows model 100 being positioned on a representation ofa floor 162 to indicate that a computing system may determine that model100 will be anchored to a floor when implemented in an AR environment.FIG. 2 illustrates upward rays 164, in dotted line, extending upwardfrom below the floor 162 to illustrate that the computing system may, insome embodiments, omit one or more rays that have a directionalcomponent that is in the same direction as a normal vector associatedwith an environmental surface (in this case, floor 162). In this regard,the system may omit from calculation rays associated with one or moreenvironmental surfaces (e.g., surfaces to which the model 100 will beanchored in an AR environment). By omitting one or more rays, the systemmay reduce the computational costs associated with performing ray castrendering. In other implementations, rather than omitting rays, thesystem adds an environmental surface as an additional constraint on theray cast rendering, such as by treating a portion of the model thatwould abut or be occluded by an environmental surface as an occlusiveplane when performing the ray cast rendering. For example, a system maydetermine that an opening of a 3D model of a glove will be occluded whenpresented in an AR environment on a user's hand. In response, the systemmay treat the opening of the glove as an occlusive surface for purposesof ray cast rendering.

Omitting rays and/or implementing an occlusive plane/surface may preventrays from reaching identified points of the 3D model that wouldotherwise would, and therefore may allow for additional mesh sections ofthe 3D model 100 to be removed, as discussed in more detail hereafter.

Those skilled in the art will recognize that any surface or plane,whether stationary, moving, flat, curved, environmental, or part of themodel 100 itself, may provide the basis for omitting one or more raysfrom ray cast rendering. For example, certain 3D objects may bepresented in an AR environment on a ceiling, wall, or distanced far fromthe user with a predefined portion of the 3D model facing the user(e.g., a 3D model of a distant mountain). In such instances, thecomputing system may omit rays that share a directional component with aviewing plane from which the 3D model will not be viewed in an ARimplementation, or implement an occlusive plane into the ray castrendering to account for viewing constraints that will be realized whenthe model is presented in an AR experience.

It should also be noted that the computing system may selectively omitportions of the 3D model from ray cast rendering and/or the occlusionreduction process altogether, such that the system does not perform raycast rendering or the on the mesh sections of the omitted portions ofthe 3D model. By way of non-limiting example, the computing system maydetermine that certain portions of the 3D model are transparent and aretherefore likely intended to be viewed when implemented in an ARenvironment (e.g., windows). Transparent surfaces will typically notfully occlude light from transmitting through the transparent surface,and so transparent surfaces may be omitted without disrupting theaccuracy of ray cast rendering. The system may then omit such portionsof the 3D model from ray cast rendering.

The system may selectively enable or disable specular reflection whenperforming ray cast rendering, and, in some instances, specularreflection is only enabled for certain model or environmental surfaces.

The system calculates an occlusion value for each of the identifiedpoints upon which the system performs ray cast rendering. The occlusionvalue for each identified point may correspond to the number of raysthat reach (or fail to reach) the particular identified point. In someinstances, the occlusion value is represented as a percentage of thenumber of rays cast (e.g., total rays cast, or rays cast from particulardirections, positions, and/or sources) that reach (or fail to reach) theidentified point. In other instances, the occlusion value is treated asa binary value, registering as “1” if any ray reaches the identifiedpoint and “0” if otherwise (or vice versa). Those skilled in the artwill recognize that the form for representing an occlusion value willvary across different implementations. For simplicity in explanation,the remainder of this disclosure will treat an occlusion value as highif a large number of rays are occluded from reaching an identifiedpoint, and low if otherwise (e.g., if a large number of rays reach theidentified point).

FIG. 2 shows several rays 160 reaching center point 144 of mesh section120. Accordingly, center point 144 will have a low occlusion value thatindicates that center point 144 is not occluded and would be visible toa user in an AR implementation. FIG. 2 shows that no rays 160 reach anyvertex points 130 of mesh section 124 (e.g., because it is covered bytop element 102), and so mesh section 124 will have a high occlusionvalue indicating that vertex points 130 are completely occluded andwould be invisible to a user in an AR implementation. FIG. 2 also showsthat, as to mesh section 126, no rays reach vertex point 146 or centerpoint 150, but at least some rays reach edge point 148. Accordingly,vertex point 146 and center point 150 will have high/maximum occlusionvalues, while edge point 148 will have a lower occlusion value.

In the implementation shown in FIG. 2, the system omits rays 164 fromthe ray cast rendering because they are associated with an environmentalsurface (i.e., the floor 162). In some instances, the system identifieswhether rays (e.g., rays 164) share a directional component with anormal vector of an environmental surface (e.g., floor 162) againstwhich the model 100 will be displayed in an AR implementation (e.g.,where the model 100 will be displayed within a threshold proximity tothe environmental surface in an AR implementation). For example, asystem may identify that the model 100 will be displayed within athreshold proximity to the floor 162 in an AR implementation, and/orthat rays 164 share a directional component with a normal vector 166 ofthe floor 162. In particular, the system may identify that rays 164 andnormal vector 166 both have a component in the positive y direction(e.g., as indicated by the axis shown in FIG. 2). The system may treatfloor 162 as an occlusive surface within the environment that preventsrays 164 from reaching the bottom of the model 100. Accordingly, no rays164 reach edge center points 142 of mesh section 122, giving edge centerpoints 142 a high/maximum occlusion value. This result differs from whatwould occur if the system did not omit rays 164 or treat floor 162 as anocclusive surface, which would result in treating edge center points 142as being unoccluded and therefore resulting their persistence in an ARimplementation (thus unnecessarily increasing the final file size bymaintaining points/mesh sections that would not be visible to a user inan AR implementation).

A system may also omit rays that are associated with surfaces of othermodels or 3D objects, such as rays that share a directional componentwith a normal vector of a surface of another virtual object that will bepresented within a threshold proximity to the model 100 in an ARimplementation (or in any 3D graphics implementation).

Upon determining the occlusion values for the various identified pointsof the various mesh sections, the system determines whether theocclusion values meet or exceed a predetermined occlusion thresholdvalue. The predetermined occlusion threshold value may take on variousforms, such as a predefined number of rays that fail to reach a point, athreshold percentage of rays that fail to reach a point, and/or anidentified binary value corresponding to total occlusion (e.g., a valueof 1 for total occlusion). In some implementations, the system comparesthe occlusion value for each identified point with the predeterminedocclusion threshold value to determine whether each identified point isoccluded or not. For instance, if an occlusion threshold is set as abinary requiring absolute occlusion for a point to be considered ashaving met or exceeded the occlusion threshold, a point with anocclusion value that corresponds with only one or few rays reaching thepoint will still be considered as unoccluded, having failed to meet orexceed the occlusion threshold. In other instances, a similar pointwould be considered occluded if the occlusion threshold were set to onlyrequire substantial occlusion rather than complete occlusion (e.g., 95%occlusion). Setting an occlusion threshold that requires completeocclusion (e.g., 100% occlusion) may ensure a more accurate output modelfile by ensuring that no unoccluded mesh sections are removed from themodel (as discussed hereafter).

Referring again to the embodiment shown in FIG. 2, if, for example, thesystem is configured to require complete occlusion for an occlusionvalue to meet or exceed the predetermined occlusion threshold value,center point 144 of mesh section 120 and edge center point 148 of meshsection 126 will be considered unoccluded, and vertex points 130 of meshsection 124, vertex point 146 and center point 150 of mesh section 126,and edge center points 142 of mesh section 122 will be consideredoccluded.

Upon determining whether each point is occluded (e.g., by comparing theocclusion values of the various points with the predetermined occlusionthreshold), the system may determine a mesh section occlusion value foreach mesh section with points whose occlusion values were assessed bythe system. In some instances, a mesh section occlusion value isdetermined based on the number of points of the particular mesh sectionthat have been determined to be occluded (e.g., a percentage or ratio).In some instances, more weight is given to certain points of a meshsection when determining a mesh section occlusion value (e.g., centerpoints can be given more weight). For example, returning to FIG. 2, allpoints associated with mesh section 120 will be considered unoccluded,and so mesh section 124 will have a low/zero mesh section occlusionvalue. Points associated with mesh sections 124 and 122 will beconsidered occluded, and so mesh sections 124 and 122 will have ahigh/maximum mesh section occlusion value. Some points associated withmesh section 126 will be considered occluded (e.g., vertex point 146 andcenter point 150), while others will be considered unoccluded (e.g.,edge center point 148), and so mesh section 124 will have anintermediate mesh section occlusion value.

The system may then determine whether the mesh section occlusion valuesof the various mesh sections meet or exceed a predetermined mesh sectionocclusion threshold. In some embodiments, the system removes meshsections from the model that have mesh section occlusion values thatmeet or exceed the predetermined mesh section occlusion threshold. Apredetermined mesh section occlusion threshold may represent totalocclusion (e.g., a mesh section occlusion value of 100%) or representany other amount of occlusion (e.g., 80% occlusion). Referring again tothe embodiment shown FIG. 2, with a predetermined mesh section occlusionthreshold that requires total occlusion, the mesh section occlusionvalue of mesh section 120 is low and will therefore not meet thethreshold. Accordingly, the system will not remove mesh section 120 fromthe model 100 as part of the occlusion reduction process. In contrast,mesh sections 124 and 122 have maximum mesh section occlusion values andwill therefore meet the mesh section occlusion threshold. Consequently,the system will remove mesh sections 124 and 122 from the model 100,thereby reducing the size of the model and improving AR implementationson various AR platforms. The system will also remove mesh section 126from the model 100 because at least some of its points are unoccluded,which brings the mesh section occlusion value below the maximum andbelow the mesh section occlusion threshold (requiring, in this example,total occlusion). However, in other embodiments, a system may removemesh section 126 if its mesh section occlusion value meets a meshsection occlusion threshold that does not require complete occlusion(e.g., a mesh section occlusion value of 66%).

It will be appreciated that the system may determine whether to remove amesh section based on occlusion values in other ways as well. Forinstance, in other embodiments, instead of first checking each occlusionvalue of each point against a point-level occlusion threshold, thesystem determines a mesh section occlusion value based on a combinationof the occlusion values of the identified points of the particular meshsection (e.g., a weighted combination, giving more weight to certainpoints such as, for example, center points) and then determines whetherthe mesh section occlusion value meets or exceeds a mesh sectionocclusion threshold.

In some instances, one or more vertices of a model may be orphaned bythe removal of mesh sections according to the occlusion reductionprocess described hereinabove. Accordingly, in some implementations, theocclusion reduction process also includes a step for removing verticesthat are orphaned by the removal of mesh sections from the model.

FIG. 3 illustrates a conceptual representation of performing ray castrendering on various points of the model 100 when the model is in adifferent state. In some instances, a model file comprises an animationin which a 3D model undergoes a change of state. FIG. 3 represents anembodiment in which the furniture model 100 is in an open state, incontrast with the closed state shown in FIGS. 1 and 2. When a model 100will be presented in an AR environment in multiple states, in someimplementations, a system performs ray cast rendering on each of theidentified points of the various mesh sections in each of the states inwhich the model 100 will be presented to determine occlusion values foreach of the identified points in each of the states. The system may thenproceed with steps outlined hereinabove, such as determining whether thevarious occlusion values of the various points in the various statesmeet or exceed an occlusion threshold, determining mesh sectionocclusion values in the various states and checking the same against amesh section occlusion threshold, etc. In some implementations, a systemonly removes a mesh section from a model upon determining that each meshsection occlusion value exceeds the mesh section occlusion threshold ineach state in which the model will be presented.

Referring again to FIG. 3, whereas the points of mesh sections 124 and122 were occluded in the model 100 states shown FIGS. 1 and 2, thepoints of mesh sections 124 and 122 are exposed when the model 100 is inthe open state shown in FIG. 3 (e.g., at least some rays 160 are able toreach the points, as shown). Similarly, while mesh section 126 was onlypartially unoccluded in the model 100 states shown in FIGS. 1 and 2, thepoints of mesh section 126 are all unoccluded when the model 100 is inthe open state shown in FIG. 3 (e.g., at least some rays 160 are able toreach the points, as shown). Accordingly, in some embodiments, a systemwill refrain from removing mesh sections 124 and 122 from model 100because in at least some states, these mesh sections are unoccluded.

Accordingly, by performing an occlusion reduction process, a computingsystem may optimize a model of a model file for viewing in an AR sceneor model, based on how the model will be viewed in the AR scene ormodel. This may reduce the file size of AR scene or model files,allowing lower-end systems to still enjoy AR content. For instance, intypical scenarios, when 3D models in CAD files are implemented into anAR environment, mesh sections for surfaces of the 3D model that will notbe visible in the AR environment (e.g., threads of a screw that isscrewed into a part) are carried into and processed in the ARenvironment, resulting in an unnecessarily large AR scene or model fileand requiring unnecessary model processing by the AR platform. Accordingto the occlusion reduction process described hereinabove, in someinstances, mesh sections that will not be visible to the user (e.g.,threads of a screw that is screwed into the part) become omitted fromthe model file, and are thus not carried into an AR model or scene filefor display by an AR platform, saving on file size and computingresources.

In addition to an occlusion reduction process, a system may performother optimization processes for optimizing a model of a model file. Atleast some optimizations may be based on conditions present in ARimplementations (e.g., which may also be applicable in conventional 3Dmodel viewing/editing implementations on a general-purpose computingsystem). For instance, because AR model or scene files are displayed byAR platforms with reference to the user's real world, real-worlddimensions may be determined/known for a model that will be displayed inan AR implementation. Furthermore, many users experience models of an ARexperience as though the models are positioned at a typical viewingdistance from the user. For instance, AR models of mountain ranges maybe comfortably viewed from a large viewing distance in AR experiences,while AR models of large-frame portraits may be typically viewed from anintermediate viewing distance in AR experiences, and AR models ofjewelry may be desirably viewed from a close viewing distance in ARexperience.

Many low-end AR devices (e.g., smartphones) display AR content overlaidon a representation of the user's real-world environment captured by oneor more cameras. The real-world environment and the AR content are bothdisplayed to a user on a screen with a known pixel resolution (e.g., apixel resolution of the AR device). FIGS. 4A and 4B illustrateconceptual representations of a real-world distance (RWD) associatedwith a viewing distance (D) and a display device 410 of known pixelresolution. The viewing distance D is a distance that would existbetween the display device 410 (e.g., a camera, screen, or anothercomponent or portion of the display device 410) and a virtual model 420being displayed by the display device 410 if the virtual model 420 waspresent in the user's real-world environment. Because the display device410 presents virtual model 420 as though it was in a real-worldenvironment, real-world dimensions for presenting virtual model 420 canbe identified. Accordingly, each pixel of a display screen of displaydevice 410 will display a representation of a portion of the virtualmodel 420 when the virtual model is being displayed by the displaydevice 410. The size of the portion of the virtual model 420 representedby each individual pixel of the display device 410 changes based on theviewing distance between the display device 410 and the virtual model420.

For example, FIG. 4A conceptualizes a situation in which the displaydevice 410 is displaying the virtual model 420 as though the virtualmodel 420 were positioned in close real-world proximity to the displaydevice 410 at a distance (D). A pixel 412 of display device 410represents all features of a certain portion of the virtual model 420,the size of the portion being represented by a real-world distance(RWD). The size of the portion may be related to the pixel resolution ofdisplay device 410 and/or the distance D. As the distance D growslarger, as conceptually represented in FIG. 4B, the RWD increases.Furthermore, in some instances, display devices with a higher screenpixel resolution may have a lower RWD associated therewith than otherdisplay devices with a lower screen pixel resolution for the samedistance D.

Because a typical pixel 412 only display one color at a time, featuresof model 420 that are smaller than the RWD may not be resolvable orrepresentable by the pixel 412, and therefore imperceptible to a userviewing a rendering of model 420 on a screen of display device 410.Allowing imperceptible features to remain in 3D model files used in ARexperiences, therefore, unnecessarily bloats AR model or scene files andcauses AR platforms to wastefully allocate processing resources. This isparticularly important for large viewing distance situations since theRWD increases with viewing distance, increasing the section of the modelapportioned to each pixel.

Accordingly, in some embodiments, a computing system performs aselective decimation step to eliminate imperceptible features from amodel that may be used in AR scenes or model files. FIGS. 5A and 5Billustrate a conceptual representation of selective decimation beingperformed on several triangles of a model 500. Selective decimationinvolves, in some instances, identifying a real-world distance threshold(e.g., RWD shown in FIG. 5A). In some instances, the RWD threshold isbased on a screen pixel resolution of a display device that will displaymodel 500 and/or a predefined viewing distance (or range of viewingdistances) between a display device and the model 500 in an ARenvironment, as described hereinabove with reference to FIGS. 4A and 4B.It should be noted that a viewing distance may be determined based onthe size or other attributes of the model (e.g., a mountain range mayhave a greater comfortable viewing distance than a piece of jewelry, aceiling tile may have a greater estimated viewing distance than a sofa,etc.), the level of detail and the general closeness of the vertices inthe model, and/or based on user data indicative of how close userstypically view the model in an AR environment, for example. An RWDthreshold may also be based simply on a percent size of the overallmodel or be subjectively predefined by a developer.

Because the model 500 will, in some instances, be displayed in areal-world environment (e.g., as part of an AR experience), real-worlddimensions may be applied to the model 500, and a system may determinereal-world distances between the vertices of the model 500. Forinstance, referring to FIG. 5A, a system may determine that a distancebetween vertex 502 and vertex 504 is less than the RWD threshold.Accordingly, a pixel of a display device will be unable to fullyrepresent the features between vertices 502 and 504. Therefore, inresponse to determining that the distance between vertices 502 and 504is below the RWD threshold, the system may merge vertices 502 and 504,as shown in FIG. 5B. Merging vertices 502 and 504 reduces the overallnumber of mesh sections that will be saved as part of the model, whichmay reduce the file size of 3D and/or AR model or scene files and allowfor faster processing, particularly in AR implementations.

Although the foregoing description of selective decimation focuses, insome respects, on AR implementations, those skilled in the art willrecognize, in view of the present disclosure, that the principlesdescribed herein may be applicable under any 3D modeling circumstance inwhich a viewing distance to a virtual object and a pixel resolution forviewing a virtual object are established. It should also be noted thatthe “real-world distance” described herein is only designated as suchfor ease of description. As used herein, a real-world distance maycomprise any distance measurement whether associated with a real-worldspace or a virtual space.

FIGS. 5A and 5B also illustrate conceptual representations of a texturereduction process. For instance, the arrangement of triangles shown inFIG. 5A may represent a u-v coordinate map, mapping the texture thatwill be applied to the various mesh sections of model 500 (e.g., meshsections associated with vertices, such as vertices 502 and 504). Theu-v map configuration may be updated to accommodate portions that areadjusted by the merging of vertices 502 and 504, resulting in a u-v mapthat corresponds to FIG. 5B.

Additionally, pursuant to a texture reduction process, a computingsystem may identify one or more features of one or more texture sectionsof the u-v coordinate map that include features that are smaller thanthe identified RWD distance. The computing system may then adjust theone or more texture sections to eliminate waste features that will notbe perceptible to a user viewing the model 500. Adjustments may include,but are not limited to, removing features, replacing features withsimpler ones, and/or adjusting texture resolution.

In this regard, pursuant to a texture reduction process, a system mayidentify a u-v map associated with a texture of the model comprising aplurality of map points associated with vertices of the model, determinea real-world distance threshold (e.g., RWD represented in FIG. 5A) and,in response to determining that two of the map points of the pluralityof map points are associated with vertices that are positioned closertogether than the real-world distance threshold, modifying one or morefeatures of the u-v map that are between the two map points (e.g., byremoving features, replacing features with simpler features, adjustingtexture resolution, etc.).

In some implementations, an occlusion reduction process is moreprocessing-intensive than other optimization techniques, such asselective decimation, in particular because of the ray cast renderingprocesses described hereinabove. Accordingly, in some embodiments,because selective decimation may result in the number of mesh sectionsof a model being reduced, a system performs selective decimation beforeocclusion reduction to reduce the number of mesh sections that thesystem must perform ray cast rendering on.

Additionally, because selective decimation and texture reductionprocesses may, in some instances, depend on the same RWD value,selective decimation and texture reduction processing may be performedsimultaneously (e.g., in parallel) to save on processing time and aspart of an automated process.

FIG. 6 illustrates an example of a 3D model 600 with model objects 660that include identical vertices (shown as identical legs of a table).Because vertices are identical for model objects 660, data associatedwith each of the model objects 660 may be merged into a single buffersuch that the model file only expresses data that is shared across themodel objects 660 once (rather than redundantly). This technique may bereferred to as “instancing.”

In addition to the foregoing, those skilled in the art will appreciatethat any number of the optimization techniques and processes disclosedhereinabove may be practiced in any combination or order. Furthermore, asystem may also perform other optimization techniques in addition tothose discussed hereinabove. For example, the foregoing optimizationprocesses may be used in conjunction with conventional geometriccompression techniques to provide a further compressed model file.

Additionally, a system may perform a texture formatting step, whereinthe system identifies one or more features of a texture of the model andintelligently determines an optimal image format based on the featuresof the texture. The system then converts the texture of the model intothe optimal image format. For example, a system my detect that a PNGimage file does not have an alpha channel or does not use an alphachannel associated therewith. In response to so detecting, the systemmay resave the PNG image file without the alpha channel to reduce itssize. In other examples, a system may determine that a texture includeslarge sections of a single color, with little color complexity, andconvert the texture into an optimal image format (e.g., PNG rather thanJPEG). In yet other examples, a system may determine that a textureincludes complex, realistic combinations of colors and/or shading andconvert the texture into an optimal image format (e.g., JPEG rather thanPNG).

Additionally, or alternatively, a system may format a texture based onan identified intended and/or predetermined use for the texture that isassociated with the model (e.g., a texture that is configured fordisplay on a mesh of the model). For example, a system may determine atype of map associated with the texture (e.g., a normal map, a diffusemap, an emissive map, an opacity map, a specular map, a gloss map, etc.)and format the texture based on the type of map associated with thetexture. In some implementations, some image formats may be optimal forrepresenting a particular map type, whereas some image formats may beill-suited for representing a particular map type. By way ofillustration, in some instances, performing JPEG compression may giverise to compression artifacts (e.g., compression rectangles) that maydistort normal maps. However, JPEG compression artifacts are oftenimperceptible in diffuse maps. Accordingly, in some instances, a systemmay determine an intended or predetermined use of an identified texture(e.g., as a normal map or as a diffuse map) and select an appropriatetexture format so as to avoid visible artifacts.

For example, a system may determine that a texture associated with amodel (e.g., a texture that will be presented on a mesh of the model)will be used as a normal map (or another bump map, in some instances).After making such a determination, the system may, in some instances,preclude the texture from being converted into a JPEG format (e.g., toavoid artifacts), even where the JPEG format might otherwise be suitablefor depicting the features represented in the texture. In anotherexample, a system may determine that a texture associated with a modelwill be used as a diffuse map (or another color map, in some instances).After making such a determination, the system may, in some instances,permit the texture to be converted into the JPEG format (e.g., becauseartifacts may be imperceptible).

In other instances, a system performs a step of clearing custom normalvectors of vertices and replacing them with an automatic normal, therebyallowing at least some of the vertex data to be merged/shared among thevertices.

In other instances, the system applies conventional decimationtechniques, but intelligently determines the optimal decimationtechnique based on attributes (e.g., concavity, curvature, polygondensity, etc.) of the portion of the model that will be decimated.

In yet other instances, the system performs acts of re-meshing andnormal map rebaking, wherein the system constructs a low polygon modelfrom a high polygon model and bakes a new normal map texture from thedifference between the initial high polygon model and the low polygonmodel. The system also rebakes other maps, such as diffuse maps, normalmaps, metallic and/or roughness maps, etc.

Upon completing all desired optimization techniques, the system outputsan optimized model file (e.g., in the form of a 3D model file, or an ARmodel or scene file). In some implementations, the system also convertsthe optimized model file into a plurality of AR model or scene fileformats that are configured for display on different AR platforms.

Accordingly, at least some of the systems and methods disclosed hereinall for low-end AR platforms (e.g., handheld devices with limitedhardware capabilities) to render AR experiences that include detailed 3Dmodels. Additionally, at least some optimization methods of the presentdisclosure account for the real-world environments in which a user willexperience the 3D models of the AR content, providing optimizationstechniques that are uniquely tailored to AR experiences. Furthermore, atleast some optimization techniques disclosed herein allow for reductionin the file size of 3D model files and/or AR model or scene fileswithout sacrificing visual fidelity. Disclosed embodiments may alsoallow AR content developers to confidently develop AR content for use onlower-end AR platforms (e.g., smartphones), thus opening new markets andallowing users to share AR experiences peer-to-peer without concern thatan AR experience will be unrenderable on a peer's AR platform.

The following discussion now refers to a number of methods and methodacts that may be performed. Although the method acts may be discussed ina certain order or illustrated in a flow chart as occurring in aparticular order, no particular ordering is required unless specificallystated, or required because an act is dependent on another act beingcompleted prior to the act being performed.

FIG. 7 illustrates an example flow diagram 700 that depicts various actassociated with optimizing a model file, according to implementations ofthe present disclosure. Act 702 is performing an occlusion reductionprocess, act 704 is performing a selective decimation process, act 706is performing a texture reduction process, act 708 is performing atexture formatting process, and act 710 is performing an instancingprocess. Each of the acts represented in flow diagram 700 may includesub-acts associated therewith, which are illustrated hereinbelow withreference to FIGS. 8-12.

Those skilled in the art will recognize, in view of the presentdisclosure, that various embodiments of the present disclosure mayimplement any number of the various acts depicted in flow diagrams 700,800, 900, 1000, 1100, and/or 1200, such as one, two, or more of thedepicted acts. Relatedly, some embodiments of the present disclosureomit one or more of the acts depicted in flow diagrams 700, 800, 900,1000, 1100, and/or 1200. Furthermore, it should be noted that thevarious acts depicted in flow diagrams 700, 800, 900, 1000, 1100, and/or1200 may be performed in addition to other optimization/compressiontechniques.

FIG. 8 illustrates an example flow diagram 800 that depicts various actsassociated with performing an occlusion reduction process (i.e., act 702from flow diagram 700).

Act 702A of flow diagram 800 is obtaining a model file that includes amodel. The model file comprises any type of AR model or scene file or 3Dmodel file that includes 3D models and/or objects, such as, by way ofnon-limiting example, .usdz, .reality, .gltf, .glb, .obj, .fbx, .dae,.sfb, 3DS, A3D, COB, SCN, C4D, DAE, DXF, IGES, IV, FBX, KMZ, LW, LWB,LWM, LWO, LWS, LXO, MA, MB, MAX, OBJ, PLY, SKP, STEP, STL, WRL, WRL97,VRML, X, X3D, or XS formats.

Act 702B of flow diagram 800 is omitting one or more transparent objectsassociated with the model. In some instances, transparent objects of amodel are intended to at least partially transmit light to provide viewstherethrough when implemented into an AR environment (e.g., windows).Accordingly, in instances where transparent objects are not intended tobe occlusive, transparent objects may be omitted from the model forperforming the occlusion reduction process (e.g., for performing raycast rendering as described herein).

Act 702C of flow diagram 800 is identifying one or more points on a meshsection of the model. In some instances, the mesh sections aretriangular sections, but any polygonal and/or curved sections (orcombinations thereof) are within the scope of this disclosure. In someinstances, the one or more points of the mesh section identifiedaccording to act 702C are points along one or more edges of the meshsection, points on a surface of the mesh section (e.g., on a center ofthe mesh section), and/or points on a corner of the mesh section.

Act 702D of flow diagram 800 is determining an occlusion value for eachof the one or more points by performing ray cast rendering. In someinstances, ray cast rendering includes casting various rays (e.g.,representative of ambient light) toward the identified points fromvarious angles to determine whether the rays reach the identified pointsor whether the rays are occluded by other parts of the model. Theocclusion value for each of the one or more points may be determinedbased on the number of rays that reach (or fail to reach) each of theone or more points (e.g., represented as a ratio or percentage, numberof rays, binary value indicating whether any rays reached the point,etc.).

In some implementations, the ray cast rendering process omits certainrays from processing, such as rays that are associated withenvironmental surfaces (or surfaces of other 3D objects). For example, aray cast rendering process may omit rays that share a directionalcomponent with a normal vector of an environmental object (or another 3Dobject) that the model will be presented within a threshold proximity to(e.g., in an AR implementation, or in any 3D graphics implementation).

Act 702E of flow diagram 800 is determining that the one or moreocclusion values meet or exceed a predetermined occlusion thresholdvalue. In some instances, a system applies a point-level occlusionthreshold to each occlusion value for each point and determines a meshsection occlusion value based on whether, how many of, and/or which ofthe points of the mesh section met or exceeded the point-level occlusionthreshold (e.g., certain points can be given more weight, such as centerpoints). The system may then, in some implementations, determine thatthe mesh section occlusion value meets or exceeds a predetermined meshsection occlusion threshold value.

In other instances, the mesh section occlusion value is determined basedon a combination (e.g., a weighted combination) of the occlusion valuesfor the one or more points of the mesh section, without regard towhether the occlusion values for the various points meet or exceed apoint-level occlusion threshold. The predetermined occlusion thresholdvalue may be selected and/or varied based on various factors, such as anintended or predetermined use for the model and/or a representation of adesired tradeoff between compression ratio and model accuracy.

Act 702F of flow diagram 800 is removing the mesh section from themodel.

Removing mesh sections from the model may reduce the file size of themodel file of which the model is a part (e.g., AR model or scene file or3D model/object file). Furthermore, act 702G of flow diagram 800 isremoving one or more vertices of the model that are orphaned by removalof the mesh section from the model.

In some instances, a model of a model file can have multiple statesassociated therewith (e.g., where the model is configured to undergo atransformation, movement, and/or other animation). Thus, decision block702H involves determining whether the model is associated withadditional states. If the model is associated with additional states,act 7021 is identifying the next state of the model, after which any ofacts 702B-702G may be repeated for the identified next state of themodel. If the model is not associated with additional states, processingmay proceed to end 702J.

Although acts 702B-702G are described with reference to a single meshsection, those skilled in the art will recognize, in view of the presentdisclosure, that references herein to a mesh section can refer to one ormore mesh sections, and references herein to any singular object canrefer to any number of that singular object. Similarly, referencesherein to multiple objects can refer to a single object.

FIG. 9 illustrates an example flow diagram 900 that depicts various actsassociated with performing a selective decimation process (i.e., act 704from flow diagram 700). Act 704A of flow diagram 900 is obtaining amodel file that includes a model. In some implementations, act 704A isperformed similar to act 702A described hereinabove.

Act 704B of flow diagram 900 is identifying a real-world distancethreshold. In some instances, the real-world distance threshold is basedon a predefined viewing distance or range of distances for viewing themodel. The real-world distance threshold may also be based on a screenpixel resolution (or range of screen pixel resolutions) that will beused to view the model in implementations (whether in AR implementationsor other 3D model implementations).

Act 704C of flow diagram 900 is determining that a distance between twovertices of the model is below the real-world distance threshold. Insome instances, a pixel of a display device for viewing the model (e.g.,at the predefined viewing distance or within the range of viewingdistances) is unable to fully represent features between vertices thatare closer together than the real-world distance threshold.

Accordingly, in response to determining that the distance between thetwo vertices of the model are below the real-world distance threshold, asystem may perform act 704D of flow diagram 900, which is merging thetwo vertices together. Merging the two vertices together may reduce thefile size of the model file of which the model is a part (e.g., AR modelor scene file or 3D model/object file).

FIG. 10 illustrates an example flow diagram 1000 that depicts variousacts associated with performing a texture reduction process (i.e., act706 from flow diagram 700). Act 706A of flow diagram 1000 is obtaining amodel file that includes a model. In some implementations, act 706A isperformed similar to act 702A described hereinabove.

Act 706B of flow diagram 1000 is identifying a u-v map associated with atexture of the model comprising a plurality of map points associatedwith the vertices of the model. In some instances, the u-v map maps thetexture that will be applied to the various mesh sections of the model(e.g., mesh sections with vertices that correspond to the map points).

Act 706C of flow diagram 1000 is identifying a real-world distancethreshold. As noted hereinabove with reference to act 704B, thereal-world distance threshold may be based on a predefined viewingdistance or range of distances for viewing the model. The real-worlddistance threshold may also be based on a screen pixel resolution (orrange of screen pixel resolutions) that will be used to view the modelin implementations (whether in AR implementations or other 3D modelimplementations).

Act 706D of flow diagram 1000 is determining that two of the map pointsof the plurality of map points are associated with vertices that arepositioned closer together than the real-world distance threshold. Insome instances, a pixel of a display device for viewing the model (e.g.,at the predefined viewing distance or within the range of viewingdistances) is unable to fully represent features between map points thatare closer together than the real-world distance threshold.

Accordingly, in response to determining that the distance between thetwo map points of the plurality of map points are associated withvertices that are positioned closer together than the real-worlddistance threshold, a system may perform act 706E of flow diagram 1000,which is modifying one or more features of the u-v map that are betweenthe two map points. In some instances, modifying the one or morefeatures may include removing features, replacing features with simplerfeatures, adjusting texture resolution and/or effects (e.g., blurring),etc. In some instances, modifying the one or more features of the u-vmap that are between the two map points may reduce the file size of themodel file of which the model is a part (e.g., AR model or scene file or3D model/object file).

FIG. 11 illustrates an example flow diagram 1100 that depicts variousacts associated with performing a texture formatting process (i.e., act708 from flow diagram 700). Act 708A of flow diagram 1100 is obtaining amodel file that includes a model. In some implementations, act 706A isperformed similar to act 702A described hereinabove.

Act 708B of flow diagram 1100 is identifying an attribute of a textureof the model. In some instances, the attribute of the texture of themodel includes an intended and/or predefined use of the texture, suchwhether the texture will be applied as a normal map or a diffuse map.Additionally, or alternatively, the attribute of the texture of themodel include, in some instances, one or more features of the of thetexture, such as channels present (e.g., alpha channels), colordiversity/complexity/combinations, shading, etc.

Act 708C of flow diagram 1100 is determining an optimal image formatbased on the attribute of the texture of the model. For example, in someinstances, an optimal image format for textures that include few colorsand/or little color complexity may be the PNG format. In some instances,an optimal image format for textures with complex colors, combinationsof colors, and/or shading may be the JPEG format. Furthermore, in someimplementations, an optimal image format for a texture that will be usedas a normal map avoids the JPEG format, which can give rise tocompression artifacts. In other instances, an optimal image format for adiffuse map may include the JPEG format, in particular becausecompression artifacts may be imperceptible to viewers.

Act 708D of flow diagram 1100 is converting the texture of the model tothe optimal image format (e.g., PNG, JPEG, or another format). In someinstances, converting the texture of the model to the optimal imageformat may reduce the file size of the model file of which the model isa part (e.g., AR model or scene file or 3D model/object file).

FIG. 12 illustrates an example flow diagram 1200 that depicts variousacts associated with performing an instancing process (i.e., act 710from flow diagram 700). Act 710A of flow diagram 1200 is obtaining amodel file that includes a model. In some implementations, act 710A isperformed similar to act 702A described hereinabove.

Act 710B of flow diagram 1200 is detecting two or more model objectsthat share identical vertices. Objects that include identical verticesoften redundantly express data for the objects represented in the modelin a space-inefficient manner. Accordingly, a system may perform act710C of flow diagram 1200, which is merging the shared identicalvertices into a single buffer. In some instances, merging the sharedidentical vertices into a single buffer may reduce the file size of themodel file of which the model is a part (e.g., AR model or scene file or3D model/object file).

Computer Systems of the Present Disclosure

It will be appreciated that computer systems are increasingly taking awide variety of forms. In this description and in the claims, the term“computer system” or “computing system” is defined broadly as includingany device or system—or combination thereof—that includes at least onephysical and tangible processor and a physical and tangible memorycapable of having thereon computer-executable instructions that may beexecuted by a processor. It will be appreciated that multiple processorsthat perform different or identical processing functions may beimplemented in a single computing system, such as various combinationsof a central processing units (CPUs), graphics processing units (GPUs),and/or holographic processing units (HPUs). By way of example, notlimitation, the term “computer system” or “computing system,” as usedherein is intended to include at least personal computers, desktopcomputers, laptop computers, tablets, hand-held devices (e.g., mobiletelephones, PDAs, pagers), microprocessor-based or programmable consumerelectronics, minicomputers, mainframe computers, multi-processorsystems, network PCs, distributed computing systems, datacenters,message processors, routers, switches, XR HMDs, and/or other wearabledevices.

The memory may take any form and may depend on the nature and form ofthe computing system. The memory can be physical system memory, such asone or more hardware storage devices, which may include volatile memory,non-volatile memory, or some combination of the two. The term “memory”may also be used herein to refer to non-volatile mass storage such asphysical storage media.

The computing system also has thereon multiple structures often referredto as an “executable component.” For instance, the memory of a computingsystem can include an executable component. The term “executablecomponent” is the name for a structure that is well understood to one ofordinary skill in the art in the field of computing as being a structurethat can be software, hardware, or a combination thereof.

For instance, when implemented in software, one of ordinary skill in theart would understand that the structure of an executable component mayinclude software objects, routines, methods, and so forth, that may beexecuted by one or more processors on the computing system, whether suchan executable component exists in the heap of a computing system, orwhether the executable component exists on computer-readable storagemedia. The structure of the executable component exists on acomputer-readable medium in such a form that it is operable, whenexecuted by one or more processors of the computing system, to cause thecomputing system to perform one or more functions, such as the functionsand methods described herein. Such a structure may be computer-readabledirectly by a processor—as is the case if the executable component werebinary. Alternatively, the structure may be structured to beinterpretable and/or compiled—whether in a single stage or in multiplestages—so as to generate such binary that is directly interpretable by aprocessor.

The term “executable component” is also well understood by one ofordinary skill as including structures that are implemented exclusivelyor near-exclusively in hardware logic components, such as within a fieldprogrammable gate array (FPGA), an application specific integratedcircuit (ASIC), Program-specific Standard Products (ASSPs),System-on-a-chip systems (SOCs), Complex Programmable Logic Devices(CPLDs), or any other specialized circuit. Accordingly, the term“executable component” is a term for a structure that is well understoodby those of ordinary skill in the art of computing, whether implementedin software, hardware, or a combination thereof.

The terms “component,” “service,” “engine,” “module,” “control,”“generator,” or the like may also be used in this description. As usedin this description and in this case, these terms—whether expressed withor without a modifying clause—are also intended to be synonymous withthe term “executable component” and thus also have a structure that iswell understood by those of ordinary skill in the art of computing.

While not all computing systems require a user interface, in someembodiments a computing system includes a user interface for use incommunicating information from/to a user. The user interface may includeoutput mechanisms as well as input mechanisms. The principles describedherein are not limited to the precise output mechanisms or inputmechanisms as such will depend on the nature of the device. However,output mechanisms might include, for instance, speakers, displays,tactile output, projections, holograms, and so forth. Examples of inputmechanisms might include, for instance, microphones, touchscreens,projections, holograms, cameras, keyboards, stylus, mouse, or otherpointer input, sensors of any type, and so forth.

Accordingly, embodiments described herein may comprise or utilize aspecial purpose or general-purpose computing system. Embodimentsdescribed herein also include physical and other computer-readable mediafor carrying or storing computer-executable instructions and/or datastructures. Such computer-readable media can be any available media thatcan be accessed by a general purpose or special purpose computingsystem. Computer-readable media that store computer-executableinstructions are physical storage media. Computer-readable media thatcarry computer-executable instructions are transmission media. Thus, byway of example—not limitation—embodiments disclosed or envisioned hereincan comprise at least two distinctly different kinds ofcomputer-readable media: storage media and transmission media.

Computer-readable storage media include RAM, ROM, EEPROM, solid statedrives (“SSDs”), flash memory, phase-change memory (“PCM”), CD-ROM orother optical disk storage, magnetic disk storage or other magneticstorage devices, or any other physical and tangible storage medium thatcan be used to store desired program code in the form ofcomputer-executable instructions or data structures and that can beaccessed and executed by a general purpose or special purpose computingsystem to implement the disclosed functionality of the invention. Forexample, computer-executable instructions may be embodied on one or morecomputer-readable storage media to form a computer program product.

Transmission media can include a network and/or data links that can beused to carry desired program code in the form of computer-executableinstructions or data structures and that can be accessed and executed bya general purpose or special purpose computing system. Combinations ofthe above should also be included within the scope of computer-readablemedia.

Further, upon reaching various computing system components, program codein the form of computer-executable instructions or data structures canbe transferred automatically from transmission media to storage media(or vice versa). For example, computer-executable instructions or datastructures received over a network or data link can be buffered in RAMwithin a network interface module (e.g., a “NIC”) and then eventuallytransferred to computing system RAM and/or to less volatile storagemedia at a computing system. Thus, it should be understood that storagemedia can be included in computing system components that also—or evenprimarily—utilize transmission media.

Those skilled in the art will further appreciate that a computing systemmay also contain communication channels that allow the computing systemto communicate with other computing systems over, for example, anetwork. Accordingly, the methods described herein may be practiced innetwork computing environments with many types of computing systems andcomputing system configurations. The disclosed methods may also bepracticed in distributed system environments where local and/or remotecomputing systems, which are linked through a network (either byhardwired data links, wireless data links, or by a combination ofhardwired and wireless data links), both perform tasks. In a distributedsystem environment, the processing, memory, and/or storage capabilitymay be distributed as well.

Those skilled in the art will also appreciate that the disclosed methodsmay be practiced in a cloud computing environment. Cloud computingenvironments may be distributed, although this is not required. Whendistributed, cloud computing environments may be distributedinternationally within an organization and/or have components possessedacross multiple organizations and/or entities. In this description andthe following claims, “cloud computing” is defined as a model forenabling on-demand network access to a shared pool of configurablecomputing resources (e.g., networks, servers, storage, applications, andservices). The definition of “cloud computing” is not limited to any ofthe other numerous advantages that can be obtained from such a modelwhen properly deployed.

A cloud-computing model can be composed of various characteristics, suchas on-demand self-service, broad network access, resource pooling, rapidelasticity, measured service, and so forth. A cloud-computing model mayalso come in the form of various service models such as, for example,Software as a Service (“SaaS”), Platform as a Service (“PaaS”), andInfrastructure as a Service (“IaaS”). The cloud-computing model may alsobe deployed using different deployment models such as private cloud,community cloud, public cloud, hybrid cloud, and so forth.

Although the subject matter described herein is provided in languagespecific to structural features and/or methodological acts, it is to beunderstood that the subject matter defined in the appended claims is notnecessarily limited to the described features or acts so described.Rather, the described features and acts are disclosed as example formsof implementing the claims.

CONCLUSION

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art to which the present disclosure pertains.

Various alterations and/or modifications of the inventive featuresillustrated herein, and additional applications of the principlesillustrated herein, which would occur to one skilled in the relevant artand having possession of this disclosure, can be made to the illustratedembodiments without departing from the spirit and scope of the inventionas defined by the claims, and are to be considered within the scope ofthis disclosure. Thus, while various aspects and embodiments have beendisclosed herein, other aspects and embodiments are contemplated. Whilea number of methods and components similar or equivalent to thosedescribed herein can be used to practice embodiments of the presentdisclosure, only certain components and methods are described herein.

It will also be appreciated that systems and methods according tocertain embodiments of the present disclosure may include, incorporate,or otherwise comprise properties or features (e.g., components, members,elements, parts, and/or portions) described in other embodiments.Accordingly, the various features of certain embodiments can becompatible with, combined with, included in, and/or incorporated intoother embodiments of the present disclosure. Thus, disclosure of certainfeatures relative to a specific embodiment of the present disclosureshould not be construed as limiting application or inclusion of saidfeatures to the specific embodiment unless so stated. Rather, it will beappreciated that other embodiments can also include said features,members, elements, parts, and/or portions without necessarily departingfrom the scope of the present disclosure.

Moreover, unless a feature is described as requiring another feature incombination therewith, any feature herein may be combined with any otherfeature of a same or different embodiment disclosed herein. Furthermore,various well-known aspects of illustrative systems, methods, apparatus,and the like are not described herein in particular detail in order toavoid obscuring aspects of the example embodiments. Such aspects are,however, also contemplated herein.

The present disclosure may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. The scope of the invention is, therefore, indicatedby the appended claims rather than by the foregoing description. Whilecertain embodiments and details have been included herein and in theattached disclosure for purposes of illustrating embodiments of thepresent disclosure, it will be apparent to those skilled in the art thatvarious changes in the methods, products, devices, and apparatusdisclosed herein may be made without departing from the scope of thedisclosure or of the invention, which is defined in the appended claims.All changes which come within the meaning and range of equivalency ofthe claims are to be embraced within their scope.

What is claimed is:
 1. A system for optimizing a model, comprising: oneor more processors; and one or more hardware storage devices storingcomputer-executable instructions that are operable, when executed by theone or more processors, to configure the system to: perform an occlusionreduction process, comprising configuring the system to: identify one ormore points on a mesh section of a model; cast a set of rays toward theone or more points on the mesh section of the model; determine a numberof rays of the set of rays that reach the one or more points withoutbeing occluded by a portion of the model or a portion of another object;determine one or more occlusion values for the one or more points on themesh section based on the number of rays of the set of rays that reachthe one or more points without being occluded; determine whether the oneor more occlusion values for the one or more points satisfy apredetermined occlusion threshold value; in response to determining thatthe one or more occlusion values satisfy the predetermined occlusionthreshold value, remove the mesh section from the model; and in responseto determining that the one or more occlusion values fail to satisfy thepredetermined occlusion threshold value, refrain from removing the meshsection from the model.
 2. The system of claim 1, wherein the occlusionreduction process further comprises configuring the system to: identifyone or more transparent objects associated with the model; and omit theone or more transparent objects from the model for performing theocclusion reduction process.
 3. The system of claim 1, wherein the setof rays omits a ray that is associated with one or more portions of anenvironment or an object configured to be presented with the model. 4.The system of claim 0, wherein the ray that is associated with the oneor more portions of the environment or the object configured to bepresented with the model comprises a directional component that is in asame direction as a normal vector associated with the one or moreportions of the environment or the object.
 5. The system of claim 0,wherein the object comprises a separate model configured to be presentedwith the model.
 6. The system of claim 0, wherein the one or moreportions of the environment comprise a surface of a real-worldenvironment within which the model is configured to be presented.
 7. Thesystem of claim 0, wherein the one or more portions of the environmentcomprise a surface of a virtual environment within which the model isconfigured to be presented.
 8. The system of claim 1, wherein theocclusion reduction process further comprises configuring the system toremove one or more vertices of the model that are orphaned by removal ofthe mesh section from the model.
 9. The system of claim 1, wherein: theocclusion reduction process further comprises configuring the system to,for a separate state of the model, determine a separate number of rayscast toward the one or more points arranged according to the separatestate of the model that reach the one or more points arranged accordingto the separate state of the model without being occluded, and the oneor more occlusion values are further based on the separate number ofrays.
 10. The system of claim 1, wherein the computer-executableinstructions are further operable, when executed by the one or moreprocessors, to configure the system to: perform an instancing process,comprising configuring the system to: detect two or more model objectsof the model that share identical vertices; and merge the sharedidentical vertices into a single buffer.
 11. The system of claim 1,wherein the computer-executable instructions are further operable, whenexecuted by the one or more processors, to configure the system to:perform a selective decimation process, comprising configuring thesystem to: identify a real-world distance threshold associated withpresenting the model for viewing; determine that a distance between twovertices of the model is below the real-world distance threshold; andmerge the two vertices together.
 12. The system of claim 0, wherein thereal-world distance threshold is based on a viewing distance for viewingthe model and a screen pixel resolution of a screen for viewing themodel.
 13. The system of claim 0, wherein the computer-executableinstructions are further operable, when executed by the one or moreprocessors, to configure the system to: perform a texture reductionprocess, comprising configuring the system to: identify a u-v mapassociated with a texture of the model, the u-v map comprising aplurality of map points associated with vertices of the model; and inresponse to determining that two of the map points of the plurality ofmap points are associated with vertices that are positioned closertogether than the real-world distance threshold, modify one or morefeatures of the u-v map that are between the two map points.
 14. Thesystem of claim 1, wherein the computer-executable instructions arefurther operable, when executed by the one or more processors, toconfigure the system to: perform a texture formatting process,comprising configuring the system to: identify a texture of the modelconfigured for display on a mesh of the model; identify an attribute ofthe texture configured for display on the mesh of the model; determinean optimal image format for the texture based on the attribute of thetexture; and convert the texture for display on the mesh of the model tothe optimal image format.
 15. The system of claim 0, wherein theattribute of the texture of the model comprises a predetermined use ofthe texture.
 16. The system of claim 0, wherein the attribute of thetexture of the model comprises a feature of the texture of the model.17. The system of claim 1, wherein the model is a 3D object from (i) anaugmented reality model or scene file, (ii) a virtual reality model orscene file, or (iii) a 3D model file.
 18. The system of claim 1, whereinthe computer-executable instructions are further operable, when executedby the one or more processors, to configure the system to: output anoptimized model file comprising the model; and convert the optimizedmodel file into (i) a plurality of augmented reality object fileformats, (ii) a plurality of virtual reality object file formats, or(iii) a 3D model file format.
 19. A method for optimizing a model,comprising: performing an occlusion reduction process, comprising:identifying one or more points on a mesh section of a model; casting aset of rays toward the one or more points on the mesh section of themodel; determining a number of rays of the set of rays that reach theone or more points without being occluded by a portion of the model or aportion of another object; determining one or more occlusion values forthe one or more points on the mesh section based on the number of raysof the set of rays that reach the one or more points without beingoccluded; determining whether the one or more occlusion values for theone or more points satisfy a predetermined occlusion threshold value;and in response to determining that the one or more occlusion valuessatisfy the predetermined occlusion threshold value, removing the meshsection from the model.
 20. One or more hardware storage devices storingcomputer-executable instructions that are operable, when executed by oneor more processors of a system, to configure the system to: perform anocclusion reduction process, comprising configuring the system to:identify one or more points on a mesh section of a model; cast a set ofrays toward the one or more points on the mesh section of the model;determine a number of rays of the set of rays that reach the one or morepoints without being occluded by a portion of the model or a portion ofanother object; determine one or more occlusion values for the one ormore points on the mesh section based on the number of rays of the setof rays that reach the one or more points without being occluded;determine whether the one or more occlusion values for the one or morepoints satisfy a predetermined occlusion threshold value; in response todetermining that the one or more occlusion values satisfy thepredetermined occlusion threshold value, remove the mesh section fromthe model; and in response to determining that the one or more occlusionvalues fail to satisfy the predetermined occlusion threshold value,refrain from removing the mesh section from the model.